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I_._  INTRODUCTION 

References  [7],  [1)  develop  a  useful  method  to  study  the  asymptotic  proper¬ 

ties  as  c  >  0  and  nr  <'['<«*  for  any  real  T  of  solutions  to  stochastic  difference 

equations  of  the  form 

(1.1)  Yc  .  =  Yk  +  th  (Y'  ,£.‘  )  +  /e  g  ( YL , £*[)  +  o(e) ,  Y*  £  Rr, 

n+ln  enn  enn  n 

where  the  distributions  of  the  random  sequence  {£C}  might  depend  on  the  {yE}. 

n  n 

Such  equations  occur  frequently  in  applications.  The  methods  in  [1]  also  work 
when  £  is  replaced  by  a  sequence  c  -*■  0  as  n  +  °°  from  which  asymptotic  proper¬ 
ties  (rates  of  convergence)  of  various  forms  of  stochastic  approximations  can 
be  obtained. 

The  emphasis  in  [1]  (an  application  of  [7])  concerned  the  case  where  the 

h^  and  g^  are  smooth,  and  no  details  for  the  non-smooth  case  or  its  applications 

were  given,  nor  was  the  asymptotic  case  where  n  -*•  »,  then  £  -*■  0  treated.  This 

is  a  deficiency,  since  in  many  applications  in  communication,  control  and 

automata  theory,  the  h  and  g  might  simply  be  indicator  functions  and  the 

noise  {£  }  depend  on  {YE},  and  the  asymptotic  properties  (as  n  ■*  00 ,  then  e  ■*  0) 
n  n 

desired.  Here,  we  apply  the  basic  results  of  [7]  to  two  such  problems.  The 
two  problems  have  current  technological  importance  in  their  own  right  and  each 
has  been  the  subject  of  a  great  deal  of  work.  Our  method  often  yields  a  complete 
analysis  of  the  asymptotic  properties  under  realistic  conditions.  The  two 
problems  are  typical  of  a  wide  class,  and  they  illustrate  the  power  and 
applicability  of  the  general  technique,  as  well  as  the  method  of  applying  it 
to  concrete  problems.  In  a  sense  the  metnod  is  an  extension  with  more  complex 
memory  structure  of  the  sort  of  "slow  learning"  results  obtained  by  Norman  19], 
and  should  have  broad  applications  to  the  areas  cited  above. 


-2- 


4  E 

The  basic  type  of  result  is  the  following.  Define  Y  (•)  ,  t  (-  [0,>»)  ,  by 

£  £  £  £ 

Y  (0)  =  Yq  and  Y  (t)  =  Y.  on  [ie,ie+e).  Under  appropriate  conditions. 

Theorem  1  gives  weak  convergence  of  { Y 1  (  - )  }  in  Dr[0,»')  to  a  particular  diffusion 

process,  as  t  +  0.  Now,  let  {nfl  denott?  a  sequence  of  integers  tending  to  00 

as  £  -+  0.  For  t  >  0,  define  YC(t)  Y*  (t+tn^)  .  The  tilde  always  denotes  a 

shift  by  n^  (discrete  parameter)  or  en^  (continuous  parameter) .  By  using 

£ 

Theorem  1  but  starting  {Y  }  at  time  n  instead  of  at  time  0,  we  will  get  a 

n  £ 

great  deal  of  information  on  the  asymptotic  properties  (large  n,  small  e) . 

The  next  section  gives  some  background  material  from  [7] .  Sections  III  to  VI 
treat  a  learning  automata  approach  to  certain  problems  in  adaptive  routing  of 
telephone  calls  [2]-[3].  The  second  problem,  in  Sections  VII-VIII,  concerns 
the  asymptotic  theory  of  an  adaptive  quantizer  from  communications  applications 


[41,  [5]. 
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II.  SOME  BACKGROUND  MATERIAL 
r  r 

D  (0,“)  denotes  the  space  of  R  -valued  functions  on  (0,°")  which  are  right- 

continuous  and  have  left-hand  limits,  and  is  endowed  with  the  Skorokhod  topology 
* 

[6].  ^  denotes  the  continuous  functions  on  R  *  lO,-*)  with  compact  support 

aql  8 

and  £  •  the  subset  whose  mixed  partial  derivatives  up  to  order  nt  in  t  and  3 
in  the  components  of  x  are  continuous.  Let  (•,•),  a_^_.  (•»•),  i,j  <_  r,  be  con- 

r 

tinuous  functions  on  R  *  [0,°°)  .  Let  the  operator 


A  =  £  b  .  (x,  t)  t —  +  -  ]  a  .  . (x, t)  - — - — 
.1  3x .  2  .  • .  n  3x , 3x . 

i  x  i  j 


be  the  infinitesimal  operator  of  a  diffusion  process  X(*).  Assume  that  the 
solution  to  the  martingale  problem  (on  D  [0,»))  of  Strook  and  Varadhan  [8] 
corresponding  to  A  has  a  unique  non-explosive  solution  for  each  initial  con¬ 
dition. 

Let  b  ( • )  denote  a  function  with  values  in  [0,1],  equal  to  1  on  S„,  = 

N  N 

{x:  | x | <N } ,  equal  to  zero  in  Rr-SN+^  and  with  second  derivatives  bounded 
uniformly  in  x  and  N.  Define  {YE,N,  n^O)  by 


(2.1) 


e,N 

n+1 


ye'N  +  [eh  <ye,n,<;e)  +  /Fg  (YE'N,CE)  +  o(e)Jb  (ye'N), 
n  enn  enn  Nn 


„e,N  e 

Y_  =  Y^  xf 
0  0 


|Y  |  £  N  and  is  zero  otherwise, 


C  N  r 

and  define  Y  '  (•)  analogously  to  Y'(-).  For  purely  technical  reasons,  it  is 

€  N  N 

convenient  to  state  the  theorem  in  terms  of  {Y  '  }.  Let  A  be  the  infinitesimal 

n 

operator  of  a  (not  necessarily  unique)  diffusion  process,  denoted  by  XN('), 
and  suppose  that  its  coefficients  a^ (•,•),  b^C*,-)  are  continuous,  bounded, 
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e  N 

have  compact  support  and  equal  a(-,-),  b(-,-)  in  S  .  Suppose  that  {Y  '  (')} 

converges  weakly  to  some  such  X^(-)  as  t  *0,  for  each  N.  Then  I"7]  lY  (•)  } 
converges  weakly  to  X(’)  as  n  -*■  The  following  theorem  is  a  restatement  of 


Theorem  3  of  [7]  with  t  *=  r .  Theorem  2  of  [7]  provides  a  very  convenient 

t  N 

method  of  proving  tightness,  and  we  will  use  it  in  the  sequel.  Let  E  '  denote 

.  e  ,N  e  .  , 

expectation  conditioned  on  tY_.  ,  j<n,  j<nl. 


Theorem  1 .  Assume  the  conditions  stated  above  on  the  solution  to  the  martingale 

r  N  N 

problem  on  D  [ 0 , °° )  corresponding  to  operator  A ,  and  on  A  and  X  ( • ) .  For  each 

N ,  and  f ( • , • )  €  ,  a  dense  set  (sup  norm)  in  ^ ,  let  there  be  a  sequence 

{f  '  (-)}  satisfying  the  following  conditions:  it  is  constant  on  each  interval 

[ne , ne+e) ,  at  ne  it  is  measurable  with  respect  to  the  o-algebra  induced  by 

{Y£,N,  j<n}  and 


(2.2)  sup  E  f  (ne)  +  sup  —  E  E  f  (ne+e)-f  (ne)  <  00  , 

1  1  c  1  n 

n, e  n, e 


and  as  e  -*■  0  and  for  each  t  as  ne  -►  t, 


(2.3)  E  |  f E  ,N  (ne)  -  f(Y£,N,ne) |  -*•  0  , 


(2.4) 


E£'Nf£'N(ne+e)-fe'N(ne)  „ 

— - (|-  +  AN)f(YE'N,ne)|  »  0. 

£  at  n 


£  N  IT 

Then,  if  {Y  '  ( • )  ,  eQ>e>0 }  is  tight  in  D  [0 ,°°)  for  each  N ,  where  does  not 
depend  on  N  and  Y£(0)  converges  weakly  to  X(0),  { Y c  { * )  }  converges  weakly  to 
X(  • ) ,  the  unique  solution  to  the  martingale  problem  with  initial  condition  X(0) . 


III.  AN  AUTOMATA  PROBLEM  -  INTRODUCTION 

Narendra  [2],  [3]  and  others  have  studied  the  application  of  automata  and 

learning  theory  to  problems  in  the  routing  of  telephone  calls  through  a  multi¬ 
node  network  and  have  suggested  a  variety  of  interesting  automata  models  for 
this  application.  Under  various  assumptions  (both  explicit  and  implicit)  they 
have  stated  convergence  results  in  a  number  of  cases.  Generally,  their  results 
are  applications  of  Norman's  [9]  results  on  slow  learning.  Here,  we  take  one 
of  their  models  and  show  how  to  apply  Theorem  1  to  get  a  much  more  complete 
asymptotic  theory  (large  time)  for  small  rate  of  change  of  the  automata  behavior 
(c) ,  and  under  more  realistic  conditions.  The  case  dealt  with  here  can  readily 
be  generalized  -  as  will  be  commented  on  below.  The  example  illustrates  the 
power  and  usefulness  of  the  approximation  techniques  used  here.  The  algorithm 
should  be  considered  as  a  prototype.  It  might  not  be  the  best,  but  it  well 
serves  to  illustrate  the  method. 

The  problem  formulation.  Calls  arrive  at  a  transmitting  or  switching 
terminal  at  random  at  discrete  time  instants  n  *  0,1,2,...,  with  P{one  call 
arrives  at  nth  instant)  =  y ,  y  £  (0,1),  P{>1  call  arrives  at  nth  instant)  =  0. 
From  the  terminal,  there  are  two  possible  routings  to  the  destination,  route 
1  and  route  2,  the  ith  route  having  independent  lines  -  and  can  thus  handle 
up  to  calls  simultaneously.  Let  [n,n+l)  denote  the  nth  interval  of  time. 

The  duration  of  each  call  is  a  random  variable  with  a  geometric  distribution: 
P(call  completed  in  the  (n+l)st  interval | uncompleted  at  end  of  nth  interval, 
route  i  used)  *  A ^ ,  A^  (E  (0,1)  .  The  members  of  the  double  sequence  of  the 
interarrival  times  and  call  durations  are  mutually  independent.  It  is  possible 
to  work  with  more  general  Markovian  arrival  processes,  but  we  retain  a  simple 
structure  in  order  to  emphasize  the  main  points.  In  practice,  a  more  complex 
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network  would  occur  -  and  perhaps  cycles  might  exist,  and  a  vector  routing 
parameter  would  be  used,  one  component  per  node.  But  the  main  idea  is  similar. 

As  in  Theorem  4,  the  average  dynamics  are  used  for  the  stability  analysis. 

From  that  point  on,  the  proof  of  the  appropriate  generalization  of  Theorem  5 
would  be  quite  similar  to  the  proof  of  Theorem  5. 

The  parameter  e  will  be  used  for  the  rate  of  adjustment  of  the  routing 
automaton-  the  device  which  selects  the  route.  The  adjustment  mechanism  will 
be  defined  later.  The  routing  automaton  operates  as  follows.  For  each  fixed 
e,  let  {y£}  denote  a  sequence  of  random  variables  -  with  values  in  10,1].  In 
order  to  have  an  unambiguous  sequencing  of  events,  suppose  that  the  calls  ter¬ 
minating  in  the  nth  interval  actually  terminate  at  time  n+-^,  and  arrivals  and 
route  assignments  are  at  the  instants  0,1,2,...  precisely.  Thus  the  state  at 
time  (n+1)  does  not  include  the  calls  just  terminated  or  calls  arriving  at  (n+1) . 

Define  the  "route  occupancy  process"  Xk  =  (X£ ' 1 , XC ' 2) ,  where  XL,;£  is  the  number 

n  n  n  n 

of  lines  of  route  i  occupied  at  time  n+.  Thus,  X£,x  <  N . .  If  a  call  arrives  at 

n  -  l 

instant  n-*-l,  the  automaton  "flips  a  coin",  and  chooses  route  1  with  probability 

£  £ 

y^  and  chooses  route  2  with  probability  (1-y^) .  If  all  lines  of  the  chosen  route 

i  are  occupied  at  instant  (n+1)  ,  then  the  call  is  switched  to  route  j  (j  ?  i) . 

If  all  lines  of  route  j  are  also  occupied  at  instant  (n+1)  ,  then  the  call  is 

rejected,  and  disappears  from  the  system. 

In  a  more  realistic  situation,  the  network  would  have  many  nodes  -  not 

simply  2,  and  many  possibilities  of  routing  from  node  to  node.  The  adjustment 

algorithm  might  be  different,  but  the  problem  would  be  handled  in  exactly  the 

same  way.  The  object  is  to  adjust  the  {yC}  sequentially  (based  on  the  system 

n 

behavior)  so  that  some  desired  behavior  occurs.  In  order  to  be  specific,  we 

use  the  following  " linear-t cwurd"  algorithm  [3],  Let  denote  the  indicator 

in 


-  /  - 


of  the  event  {call  arrives  at  n+1 ,  is  assigned  first  to  route  i  and  is 
accepted  by  route  i).  For  practical  as  well  as  theoretical  purjosos,  it  is 


important  to  bound  y  away  from  the  ixsmts  0  and  1.  Let  0  <  y„  <  y  <1.  Wc 

n  f  u 

yu 

denotes  truncation  at  y  or  v,,  and  a(y)  = 
y{  u  rt' 


use  the  algorithm  (3.1),  where 
1-y,  B (y)  =  -y . 


(3.1)  y^,  =  [y£  +  ea(y£)J£  +  EB(y£)JE  ] 

n+1  Jn  •'n  In  J  n  2n 


Define  a£(-),  8  _  ( - )  such  thata(-)  =  (•)  in  [y^y^-e]  and  S(*)  = 

B£(*)  in  [y^+e  ,y  ]'  and  otherwise  are  such  that  (3.2)  is  equivalent  to  (3.1) 


(3.2) 


yj\.  =  yC  +  e[a  (y£)J£  +  B  (y£)J£  ] 

n+1  n  e  n  In  e  'n  2n 


We  will  study  the  asymptotics  of  the  behavior  of  a  centered  and  normalized 

{ y n }  for  small  e.  Part  of  the  difficulty,  which  our  scheme  is  well  able  to 

handle,  is  due  to  the  fact  that  (y£)  is  not  Markovian.  In  the  theoretical  parts 

n  - 

of  [2],  [3],  the  problem  is  set  up  so  that  {y£}  is  Markovian. 


Some  definitions.  If  the  choice  probabilities  y£  are  held  fixed  at  some 

value  y  for  all  n,  then  the  route  choice  automaton  still  makes  sense,  although 

there  is  no  learning.  For  fixed  route  selection  probability  y  E  (0,1),  let 
1  2 

(Xn(y)  }  =  tOC^ (y)  ,X^{y)),  0£n«®}  denote  the  corresponding  route  occupancy 
process.  For  the  process  (x^y)},  the  state  space  Z  «  ((i,j):  i<N  , (whose 
points  are  supposed  ordered  in  some  fixed  way)  is  a  single  ergodic  class,  and  the 
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probability  transition  matrix,  denoted  by  A'(y),  has;  infinitely  differentiable 

components.  With  qiven  initial  condition  (P(X  (y)=<t),  7, 1  define  1’^  ( a  |  y )  = 

P{X  (y)=a}  and  the  vector  P  (y)  -  {p  (a|y),  ti£  z)  .  Then 
n  n  n  1 1 


0.3)  pn+i(y)  =  A(y)pn(y>  • 


The  pair  { ( XL ,yL ) , n>0 )  is  a  Markov  process  on  X  '  ly, ,y  I  and  the  marginal 
n  n  —  c  >  u  -  •  — J - 

transition  probability  P{XE  = (k , l)  ! XC= ( i , j) ,  yE)  is  just  the  (( i , j) -column , 
- — - -  n+1  n  J  Jn 

(k,{.)-row)  entry  of  A,(yE)  .  Define  the  vector  PE  =  {pE(a),  aE  z)  where  Pt(a)  = 

n  n  n  n 


P{XE=a|yE,  £<n,  XE}.  Then 


PC..  =  A(yC)P£. 
n+1  Jn  n 


Also,  let  P(y)  =  {P(a|y),  «Ezl  denote  the  uniauo  invariant  measure  for  {X  (y)  } , 

n 

with  marginal  defined  by  P1  (  j  [  y )  =  ][  P  ( j ,k | y)  ,  P1  (y)  »  {P^  ( j  |  y)  ,  } ,  and 

similarly  for  route  2.  Finally,  define  the  transition  probability  P(o,j,a  |y)  ■= 


P{X^.  (y)  |  XQ  (y)  =a}  and  write  the  marginal  as 


P 1(a , j ,k | y)  =  P{X^(y)=k|xo(y)=a) . 


Define  EE  to  be  the  expectation  conditioned  on  (x^,  yE,  iUn) . 


A  relationship  of  (3.1)  to  a  differential  equation.  Define  «  (1-XJ 
Note  that 


e£  j!t  -  uyC  [1  -  v.Kx'  '  !,N  }  ]  , 
n  In  n  1  n  1 


(3.5a) 
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(3.5b)  e£j£  =  u(l-yc)tl  -  vt{xc'2=n  }] . 

n  2n  -'ll  2  n  2 


For  small  e,  it  is  reasonable  to  try  to  relate  the  behavior  of  {yL)  to  the 

n 

solution  of  (3.6),  where  F(y)  is  just  Ela(y)J^  +  iMyU^],  but  wi  th  {x£,y£} 
replaced  by  {x^(y),y}  and  using  the  stationary  measure. 

(3.6)  y  =  pa(y)y [l-v1P1!N1 |y) ]  -  u (1-y) 6 (y) [l-v2P2 (N2 | y) ] 

0  1  A 

=  yy(l-y)Iv  P  (N2 1 y)  -  |y)]  £  F(y). 


As  y  increases,  P^dJ  |y)  increases  (and  P2(N2|y)  decreases)  monotonically . 
Thus,  there  is  a  unique  point  y  G  (0,1)  such  that  F(y)  *  0.  Also,  F(y)  >  0 

for  y  <  y  and  F(y)  <  0  for  y  >  y.  We  assume  that  y  G  (y„ ,y  )  and  we  also  make 

~  U  " 

the  apparently  unrestrictive  assumption  that  FMy)  ?  0.  We  actually  will  study 

the  asymptotic  properties  of  U£  £  (y^-y)/^t\  for  large  n  and  small  e.  In 

particular,  let  n^  be  a  sequence  of  integers  tending  to  00  as  e  -+•  0,  and  define 

the  processes  U£(’)  by  U£  (0)  =  UE  and  U£(t)  *  (J£  on  [ie,ie+e).  When  the 

ne  c 

U£(*)  are  dealt  with,  the  {n^.}  will  either  be  explicitly  defined  or  their 
values  will  be  unimportant.  We  show  weak  convergence  of  { Ue ( * ) }  to  the  Gauss- 
Markov  diffusion  u(-)  defined  by  (6.3)  .  If  n  -*  <*>  fast  enough  as  e  •+■  0,  then 
the  limit  u(-)  is  stationary.  The  general  method  can  be  applied  to  many  other 
problems  in  learning,  automata  and  systems  theory. 


i. 
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IV.  SOME  PRELIMINARY  RESULTS  j 

t 

In  this  section,  we  prove  some  auxiliary  results  concerning  uniform  con¬ 
vergence  of  P  (y)  and  its  derivatives  to  P(y)  and  its  derivatives.  * 

Theorem  2.  For_each  y  E  A’(y)  denote  a  Markov  transition  matrix 

(continuous  in  y)  such  that  the  corresponding  Markov  chain  (x^(y))  is  ergodic  j 

with  invariant  measure  P(y) .  Then  P(*)  is  also  continuous  and  there  is  a  6  >  0 
such  that  the  eigenvalues  of  A(y) ,  except  for  the  single  eigenvalue  unity, 
are  bounded  in  absolute  value  by  1-6  for  all  yE  [y^y^].  P^(y)  converges  to 

P(y)  uniformly  (and  at  a  geometric  rate)  in  y  E  [y^,y  ]  and  in  PQ(y) •  1 

Proof.  The  last  sentence  follows  from  the  penultimate  sentence.  The 
continuity  of  P(-)  is  a  consequence  of  the  uniqueness  for  each  y,  of  the  eigen¬ 
vector  of  A (y)  corresponding  to  the  eigenvalue  unity  (the  invariant  measure) . 

Next,  suppose  that  there  is  no  such  6.  Let  A(y)  be  a  q*q  matrix  and  let  1 

X^(y),  ...,  X  (y)  denote  the  eigenvalues.  Order  them  such  that  X^fy)  H  1. 

Then  there  is  a  y  and  a  sequence  {y  }<Zfyn»y  ]  such  that  as  y  -*■  y,  at  least 

J  n  i  J  u  n 

one  eigenvalue  (other  than  the  one  which  is  always  unity)  approaches  the  unit  1  4 

circle.  In  particular,  suppose  that  the  ordering  is  such  that  jX^Cy^l  1 

and  that  (choosing  a  subsequence  if  necessary)  the  X^(y^)  converge  to  some  X^ 

as  n  ■*  00 ,  for  i  =  l,...,q.  The  { X ^ }  must  be  the  eigenvalues  of  A(y)  .  But 

then  A' (y)  is  not  the  transition  matrix  of  an  ergodic  process,  a  contradiction. 


Q.E.D. 
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Definltion.  Let  £ (y)  denote  the  span  of  the  eigenvectors  and  generalized 
eigenvectors  of  A(y) ,  except  for  the  eigenvector  which  corresponds  to  the 
eigenvalue  unity. 

Theorem  3 .  Assume  the  situation  of  Theorem  1 ,  but  let  A ( • )  be  continuously 
differentiable  on  (at  the  endpoints,  take  the  left-  or  right-hand  deri¬ 

vatives,  as  appropriate) ;  then  so  is  P  <  • )  ,  and  p  (y)  is  the  unique  solution  in 
£ (y)  to  the  equation 

(4.1)  Py(y)  =  A(y)Py(y)  +  A^(y)P(y). 

Furthermore,  the  derivative  P^  ^(y)  given  by 

(4.2)  P  (y)  =  A (y) P  (y)  +  A  (y)P  (y) 

n+1  ,y  J  J  n,y  y  1  n 

converges  geometrically  to  Py(y) ,  uniformly  in  y  £  ly^#yu3  and  in  the  initial 

condition  P_(y),  if  we  set  P„  (y)  =0. 

-  0  -1  -  0,y  1 

If  A(  •)  has  continuous  second  derivatives  on  [y ^ , yu J ,  then  so  do  P ( • )  and 
Pn(  • )  ,  and  P^ (y)  is  the  unique  solution  in  £  (y)  to 

(4.3)  P  (y)  =  A(y)P  (y)  +  2A  (y)P  (y)  +  A  (y)P(y). 

yy  yy  y  yJ  yy  * 

Also,  P^  yy(y)  converges  geometrically  to  P  (y) ,  uniformly  in  y€ 

and  in  the  initial  conditions,  if  P_  (y)  «  P„  (y)  ■  0. 
-  0,y  v  0 ,yy 
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Proof.  Fix  y.  Since  (I-A(y))V  =  0  for  V  £  >'(y)  implies  that  V  =  0,  in 
order  for  (4.1)  to  have  a  unique  solution  in  » ( y )  it  is  necessary  and  suffic¬ 
ient  that  A^(y)P(y)  J-  >f(I-A'(y)),  where  #  denotes  the  null  space  of  the 
matrix.  _/f(I-A'(y))  is  the  set  of  vectors  Q  such  that  A'(y)p  *  Q.  Since  there 
is  a  unique  eigenvalue  of  value  unity  and  since  the  row  sums  of  A'(y)  are  all 
unity,  the  components  of  Q  must  all  have  the  same  value.  Thus,  the  necessary 
and  sufficient  condition  reduces  to  A^(y)P(y)  _L  constant  vectors.  For  any 

constant  vector  C  =  (c,c,...)',  C'A(y)  =  C'.  Thus,  C'A  (y)  =  0  and  hence 

Y 

A  (y)Dl  constant  vectors  for  any  vector  D.  Consequently  (4.1)  has  a  unique 

solution  P  (y)  in  £(y). 

Y 

Next,  we  show  that  P  (y)  is  the  desired  derivative.  Write  (for 

y 

Y  €:  (y£»Yu) >  otherwise  6  >  0  or  6  <  0,  as  appropriate) 

A(y+<5)P(y+6)  -  A(y)P(y)  =  P(y+6)  -  P(y). 


Thus , 


(4.4) 


P(y+6)  =  (I-A(y))PJ^)-P^.U 

o  o 


The  left-hand  side  of  (4.4)  is  uniformly  bounded  and  is  in  E(y)  for  each  6  >  0 
(since  (I-A(yl)vG  T.  (y)  for  any  V)  and  it  converges  to  A^(y)P(y)  as  6  -*■  0.  When 
considered  as  an  operator  from  T.  (y)  to  I  (y)  ,  [I-A(y)]  has  a  bounded  inverse, 
thus,  as  i  -»  0, [P(y+6)-P(y) )/6  converges  to  P^(y) ,  which  must  equal  Py(y),  by 
the  uniqueness  proved  above. 

We  now  turn  to  the  convergence  (4.2).  By  Theorem  1,  P^(y)  converges  geo¬ 
metrically  to  r(y)  ,  uniformly  in  lYjfYy]  ®nd  i»  Pg(y).  Also,  since  we  use 

P„  (y)  -  0, 

0,y 


x 
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n 

P  _  (y)  =  T  An  1(y)A  (y)P.(y). 

n+i,y  -r0  y  i 

But  A  (y)P.(y)  is  a  bounded  sequence  in  J-(y),  and  as  i  -►  <*>  it  converges  geo¬ 
metrically  and  uniformly  to  A^(y)P(y).  Also  A(y)  is  a  contraction  when  acting 
in  }’.(y),  uniformly  in  y  G  ly^y^].  These  facts  imply  the  desired  convergence 
of  P  (y) .  The  limit  must  be  a  solution  to  (4.1)  . 

n.y 

The  assertions  concerning  P  are  proved  in  the  same  way  and  we  omit  the 


details . 


Q.E.D. 
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V.  TIGHTNESS  OF  {U£,  SMALL  e ,  LARGE  n) 

_ rr _ _ _ 

By  "e  small"  and  "n  large"  we  mean  that  there  are  e^  >  0,  Nf  <  «>,  such 

that  the  assertion  holds  for  c  <  e_,  n  >  N  .  The  actual  value  of  t  will  be 

—  0  —  e  u 

unimportant.  Basic  to  the  proof  of  weak  convergence  of  {u*  (•)}  is  the  tightness 

of  {Uc,  small  e,  large  n) . 
n 

Theorem  4 .  For  each  small  e  >  0,  there  is  an  <  00  such  that  the  doubly 
indexed  sequence  {uG,  e  small ,  n  >_  N^ }  is  tight,  where  UG  =  (y^-y)//e. 

-  2 

Proof .  Define  V(y)  =  (y-y)  .  We  have 


(5.1a) 


e ,  £ 
E  (y 
n  1 


n+1 


<>  *  *  Vy‘>u-r‘>u-v2ifx‘'2-V], 


<5.ib.  <(y^ry')2 


e2y[a2(yC)ye(l-v  l{xe,1=N1})  +  $2 (yE) (l-yE ) (1-v  I (XE ' 2=N. }) ) 
e  n  ■'n  In  1  e  n  7n  2  n  2 


For  small  e. 


Ee(yC-y)[a  (y£)JE  +B  <yE)j!:  3  <  EC  (y£-y)  (a  (yC)  JE  +6(yE)JE  ], 

n  7n  7  e  n  l,n  e  7n  2,n  -  n  7n  7  n  l,n  7n  2,n 


since  0  _<  a£(y)  _<  a(y)  and  a£(y)  /  a(y)  only  if  yE-y  >  0  (for  small  e)  ,  and  con¬ 
versely  for  the  p  term.  Using  the  above  inequality,  (5.1a)  and  |yE  ,-yG|  ■  0(e), 
c  n+1  n 

(5.2)  Eev(ye  , ) -V (yf )  <  2pe(yC-W  (a(yC)yC(l-v1l{xC'1*Nj) 
n  7n+l  n  -  Jn  n  n  1  n  1 


+  B(yE)  (l-yE)  (1-v  l{xL'2-N  ))  ]  +  0 ( e 2 )  • 
n  n  2  n  2 
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Define  VE  (n)  by 

a> 

(5.3)  (n)  «=  2\m  (y^-y)u  (y^)  y^Vj  }  lP*(N^|y*)  -  r1  (X^  ,  1-n,N^  I  y^)  ) 

j«n 

a> 

+  2i.E(y|i-y)B(yJ(l-Y^v2  l  [P2  («2!  V^)  -  P2  (x‘  ,  j-n.N  |  yE>  1  . 

j*n 

Note  that  P  (Xe  ,0,N  .  |yk' )  =  l{XE'1=N,}.  By  Theorem  2,  the  sums  converge  absolutely 
ni  n  ni 

(the  summands  go  to  zero  at  a  geometric  rate)  uniformly  in  n,  yf' ,  X1  .  Thus 

n  n 

| V*"  ( • )  |  =  0(e),  uniformly  in  all  the  variables. 

Next,  evaluate 


E^(n+l)-v[(n)  =  -2ye (yE-y) a (yE) yEv  [P1  (N.  |yG)  -  l{XE,1=N  }] 
n  1  i  n  n  n  1  1  n  n  1 


2ue(y"-?)6(y^)(l-yE)v  (P2(N  |yE)  -  l{xG,2=N  }] 
n  n  n  z  z  n  n  2 


15.41  ♦  l  2u,v  (^(y'.1-5)«ly‘,1ly^,fo1(«1lv^1)  -  )1 

j=n+l 


1  ,..E 


(yE-y)a(yE)yn(P1(N1 |yE)  -  P1 (XE , j-n,N1 | yE) ) ) 


+  a  similar  sum  for  route  2. 


2 

We  next  show  that  the  sums  in  (5,4)  =  0(e  )  uniformly  in  all  the  variables 

e  £ 

n,  y^,  Xn.  For  simplicity  we  work  only  with  the  first  sum  (route  1) .  By 

iy^+1-y^|  =  0(e),  the  smoothness  of  a(*)  and  B(-)  and  Theorem  2,  the  sum  changes 

by  0(e2)  if  (yG+1~y) a (yE+^) y^+1  is  replaced  by  (yE-y) a (yE) yE .  Upon  making  the 

substitution  and  using  the  Markov  property  of  {X^(y),  j>nl  with  the  value  y  - 

yG  and  "initial"  condition 
n 
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X  (y£)  =  XE, 
n  n  n 


e‘ P1  (XL  ,  j-n- 1 , N  | y £ ) 
n  n+1  1 '  n 


P^X^j-n.Njy") 


we  can  rewrite  the  sum  as 


(5.5)  0(e)  l  E^UP^N  |y|;+  }  '  pl(Ni!Yn)] 

j=n+l 


"  [pl(Xn+l'j_n'1'NllYn+l)  -  ,,1(Xn+l'j"n~l’Nl|yn)1}  +  0(f2) 


G  £  f  1 

Write  5y^  =  y^+^-y^,  and  use  the  differentiability  (Theorem  3)  of  the  P  and 

the  law  of  the  mean  to  write  (5.5)  in  the  form 


0<eK  .  I  < 

j=n+l 


[P1 (N  | y£+s6yE) 
y  1 1  n  n 


1  2 
P  (XE  fj-n-l,N  jye+s6y  )]ds  +  0(E  ) 
y  n+1  J  ll7n  n 


By  Theorem  3,  the  sequence  of  absolute  values  of  the  integrands  converqes  to 

zero  geometrically  as  j  -►  “ ,  uniformly  in  s,  n,  <SyC ,  and  XL  This,  together 

n  n+1 

■  e  i  2 

with  | 6y  |  =  0 ( t ) ,  yield  that  (5.5)  is  0(e  ).  The  same  result  holds  for  the 

sum  in  (5.4)  corresponding  to  route  2. 

Define  VE(n)  =  V(yE)+VE(n).  By  (5.2)  and  (5.4)  and  the  fact  that  the 
2 

sums  in  (5.4)  are  0(e  ), 


E£ve (n+1) -VE (n)  <  0(e2)  +  2ye 'yE-y)  la (yE) yE (1-v, P1 (N  | yE) ) 
n  —  n  7  n  n  1  1 1  7n 


+  3  (yE)  d-yE)  a-v2P2  (N2  |yE) )  ] . 


Otfing  to  the  definition  of  a(‘)  and  B(-)  and  the  fact  that  £  ty^y^'  the 
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bracketed  term  has  its  unique  zero  at  =  y  and  it  is  positive  (negative, 

resp.)  for  y£  <  y  (y£  >  y,  resp.).  Thus,  there  is  a  y  >  0  such  that 
n  n 

(5.6)  E£V£(n+l)-V£(n)  <  0 ( e2)  -  eyV(y£) . 

n  —  J  n 

By  jv£(n)|  =  0(r.)  uniformly  in  n,  E£V£ (n+1 ) -VC ( n)  <  0(t:2)  -  r-yVL(n),  and  hence 

(5.7)  EV£(n)  <_  (exp  -  eyn)EV£(0)  +  0(e). 

Again,  since  |V^(n) |  =  0(e),  uniformly  in  n,  (5.7)  holds  for  V(y^)  replacing 
ve(n),  from  which  the  existence  of  the  {N^}  and  the  asserted  tightness  follows. 
In  particular,  let  0  <  KQ  be  arbitrary  and  let  N£  be  the  smallest  integer  n 
such  that  (exp  -  eny)  _<  KQe. 


Q.E.D. 
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VI.  WEAK  CONVERGENCE  OF  {U  (•) 


Definition .  Recall  the  definition  of  N^  given  at  the  end  of  the  proof  of 
Theorem  4.  For  any  sequence  of  integers  n^  >  ,  define  *  n  -N  .  Define 

'Vf.  i  m 

y  =  y  and  similarly  define  the  "shifted"  sequences  U  ,  X  and  J.  .  Then 

n  n  +n  n  n  in 

e 


(6.1) 


V\  =  UE  +  ^[«  (y£)j"  +  6  (y£)3f"  ] 

n+1  n  e  n  In  f.  n  2n 


,'V£  .  %£  ,N  'I'E  ,N 

By  Theorem  4,  lU  ,  t  small}  is  tight.  For  each  integer  N,  define  U  ,  y  , 
n  n  n 

J .  via 
m 


(6.2)  S';"  -  Se-N  +  +  6  (yE  J^,N)b  (UE  ,N)  , 

n+1  n  n  In  n  2n  N  n 


O.C  N  *V£  1  | 

where  bM (•)  is  defined  above  (2.1)  and  we  set  U  -  Un  if  |U  |  <  N  and  equal 
N  U  U  0 

to  zero  otherwise.  Also  U  =  (y  -y)/*^  defines  y  .  J.  is  simply  the 

n  n  -  n  in  - 

indicator  function  of  the  set  {route  i  is  tried  first  and  call  accepted}  for 

the  system  {XE,N,yE,N},  where  the  choice  probabilities  {yE,N}  are  used  to  select  the 

routes  and  {xE,N}  is  the  corresponding  route  occupancy  process.  We  suppose 

N  £  n  've . n 

that  X  '  »  X  .  Let  E  denote  expectation  conditional  on  y  and  X 

0  n  n  n  n 

e 

Since  |y  ’  -y I  <  /e(N+1)  ,  for  small  t  it  is  irrelevant  whether  we  use  a  ,  6  or 
1  n  1  —  c  e 

a,  B  in  (6.2),  and  we  use  a,  B  for  simplicity.  By  Theorem  1,  if  we  show  that 
(for  each  N)  {U  (•) }  is  tight  and  that  all  weak  limits  satisfy  (6.3)  until  first 
escape  from  S^,  then  {&E(0}  is  tight  and  all  weak  limits  satisfy  (6.3). 

We  now  define  some  auxiliary  processes  which  are  used  in  the  averaging 
method  errdoved  in  the  proof.  Let  P  denote  the  measure  defined  bv  the  stationary 


-iroc^ss  (X.(v),  with  oorresnondinn  expectation  operator  E. 


For  each  n. 


it  is  necessary  to  introduce  the  process  { X ^ (y ) r  j^n},  but  with  "initial"  condition 


Xn(y) 


2E'N. 


(I.e.,  after  time  n,  the  route  choice  probability  is  y.)  The  opera- 
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tor  E^  denotes  the  expectation  of  functions  of  this  process  {x^(y),  j>n}  condi¬ 
tional  on  the  "initial"  condition  X  (y)  =  XL,N.  Let  J. . (y)  denote  the  indicator 

n  n  i] 

function  l{call  arrives  at  j+1,  is  assigned  to  and  accepted  by  route  i),  when  the 
route  choice  variable  is  y  and  the  route  occupancy  process  is  (x^(y)}.  Whether  we 

intend  the  ergodic  process  or  the  process  (x^(y),  i^n}  starting  at  time  n  with 

—  # N  „  _  — r  n 

X  (y)  =  X  will  be  made  obvious  by  use  of  either  E  or  E  9  .  Define 
n  n  n 


<5u^(y)  =  [a(y)J  (y)  +  B  (y)  J2_.  (y)  ] . 


Under  P,  the  right  side  has  zero  expectation. 


Theorem  5 .  For  any  sequence  nE  Ne »  (U  ( • )  )  is  tight  in  D  [0 ,°°)  .  All  weakly 
convergent  subsequences  converge  to  a  Gauss-Markov  diffusion  satisfying  (6.3) . 

If  eQc  -*■  “  as  c  -*■  then  the  limiting  diffusion  u(-)  is  stationary  in  that 
u(0)  has  the  stationary  distribution.  (In  all  cases  u(0)  is  independent  of  B(-).) 


(6.3)  du  =  Gudt  +  odB,  B(-)  =  standard Brownian  motion , 


.4)  G  =  F  (y)  =  |-  yy(l-y)  [v2P2(N2|y)-v1P1 (N1|y) ] 


(6.5)  o2  =  E(6uQ(y))2  +  2  \  E  <5uQ(y)5u  (y)  . 

n=l 


Proof.  Part  1.  Until  Part  4,  all  superscripts  N  will  be  omitted.  Thus 


,2 ;e  7X  yt  i  \  *  r£e,N  -_-e,N  ^t,N  ^e,N,i  ,  „ 

we  write  (E  ,E  , X  ,y  ,  X  ,...)  for  (E  ,E  ,X  ,y  ,X  ,...).  We 

-  n  n  n  n  n  - n  n  n  n  n  — 


actually  work  with  the  N-truncated  process  in  Parts  1  to  3. 
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By  (5.1), 


r(?J£  -££)  =  m,yE(l-yC)[v  T^'^l-vlI^'Wjlb  (UC) 

n  n+1  n  'n  *n  2  n  Jin  1  N  n 


Let  f  (■,•)£  =  i?  '  ‘  ,  the  space  of  bounded  (x,t)  functions  with  compact 

support  whose  mixed  partial  derivatives  up  to  order  2  in  t  and  J  in  x  are 

r\i  j  | 

continuous.  To  apply  Theorem  1  to  lU  (•)),  we  will  get  an  f  (•)  of  the  form 


f£(ne)  =  f(U£,ne)  +  f£(ne)  +  f£(ne)  +  f£(nc) 
n  0  1  2 


where  the  f . (nc)  will  be  defined  in  the  sequel.  For  each  N,  all  o(‘)  or  O ( - ) 
are  uniform  in  all  variables  except  their  argument.  We  have 


Etf(U£  , ,nf  +  e) -f (u‘  ,ne)  =  EL[f(UG  ,ne) -f (U£ ,nc) \  +  f  .(U£,ne)E  +  oU)  , 
n  n+1  n  n  n+1  n  t  n 


E£[f(U£  ,ne)-f(U£  ne)  ]  =  E£f  (U£  ne)  (££ +J  -U£)  +  (IJL  ,nt)  (u'£  -UL)  2  +  o(c) 

n  n+1  n  nun  n+i  n  2  n  uu  n  n+1  n 


(6.7)  =  Ztiif  (U£  ,  ne)  yE  (l-y£)b  (UE)  [v  I  [XG  '  2=N  l-v  I  {XG  '  *=N  } ) 

un  n  n  N  n  2  n  2  1  n  1 

f  'vt  ’^e  .'ve  'vc,  2 

+  -Hy(Un'nc)En(Un+l'Un)  +  °(£)* 

2 


By  the  differentiability  result  of  Theorem  3,  we  can  rewrite  the  term 
before  the  o(e)  as  follows:* 


GbN  <*n>  fa A^K  [a  ^n+6  ]  ’ 

2 


EbN<Un)fuu<Un'ne)En[a(y)Jin^)+6(y)J2n(y)]  +  °<e>  • 


*The  terms  ECJ,  (y)  and  &£J^  (yL)  differ  only  in  that  in  the  first  cass  y  is  used 
n  In  n  In  n  -\,c  -ve 

as  the  choice  variable  to  qet  the  successor  state  to  X  ,  and  y  is  used  in  the 

n  n 

second  case. 
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Part  2.  We  will  "average  out"  the  terms  in  (6.7)  one  by  one.  Define 

€ 

f^(ne)  (analogous  to  the  definition  of  V^(n)  in  the  last  section) 

a> 

(6.8)  f‘  (in  )  =  S\  Uh  l(U')yL(l-y'  )  f  (li'.nt)  )  1  v  ( r”  ( x'  , j-n , N  | y* )  - 

1  N  n  n  nun  ,L‘  n  2  n 

2  l  'Vl  ]  'V/L  I  '\»|  1  I  'W 

-  P  (N  y  ))  -  u  (P  (X  ,j-n,N  y  )  -  P  (N  y  ))]. 

2  n  1  n  I  n  in 

Proceeding  analogously  to  the  method  of  Theorem  4  for  V^(n),  we  evaluate 

(writing  P  (X  ,i-n,N.  y  )  in  the  more  convenient  form  E  P  (X  , ,j-n-l,N.  y  ) 
n  l  n  n  n+1  J  l  n 

in  below,  for  j  ^  n;  see  above  (5.6)) 

(6.9)  &ef^(ne+e)-f^(ne)  =  T.  +  T.  +  T- , 

n  1  1  12  3 

T  =  -/^uyc(l-ye)bM(Se)f  (Ue,ne)  [<v  l<xl  ,2=N  t-v  ifx^'^N  }) 

1  n  •'nNnun  2n  21n  1 

-  (v2P2(N2|y^)-vlP1(N1|y^) ) ] 
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Using  the  differentiability  result  of  Theorem  3  and  the  fact  that 
v?P  (N^jy)  =  v  P  (N^|y),  we  get  that  equals  the  negative  of  the  first  term 

on  the  right  side  of  (6.7)  plus 


(6.10)  f,pb  ,  (UC)q-(y  (1-y)  lv  P"(N  |y)  -  v  P1  (N  1 y) )UL ]  +  o(i). 

N  n  3y  2  2'  1  1'  n 

y-y 


%£.  'W  'X'C  %£ 

In  T_,  by  replacing  y  .  by  y  and  b  (U  ,«)f  (U  ,  , ne+c)  by 
2  J  n+1  J  Jn  N  n+1  u  n+] 


'Vr  'Vp  'V.r  'Vr  'X/p 

b„(U )f  (U  ,ne)  +  (h(U  )f  (U ,ne)>  (U  -U >, 
Nnun  Nnun  u  n+1  n 


we  only  alter  the  term  by  o(e) .  Let  us  make  these  replacements  in  T^  and  denote 

0  0  2  2 
the  resulting  term  by  T^.  Now,  split  T 2  into  two  parts  (T^,  T^)  ,  the  first 

(second,  resp.)  being  T  but  with  b  (UE)f  (UE,ne)  ( (b  (UE) f  (U£,ne))  (UE  ,-U£), 

2  Nnun  Nnun  u  n+1  n 

%£  'X'G 

resp.)  replacing  b  (U  ,,)f  (U  ,nc+i:)  .  By  the  differentiability  results  of 
N  n+1  u  n+1  J 

I ^E  'Vp | 

Theorem  3  and  the  fact  that  lyn+^-ynl  =  °(e'  ana  an  argument  like  that  below 
(5.5),  it  can  be  shown  that  =  o(e)  .  Thus 


(6 .  lla^  T  +  T  =  o(c)  +  /euyC (l-yC)  (b  (UE) f  (U£,ne)) 
’2  3  n  nNnun  u 


3=n+l 


Vl(pl(^n+l'j~n"1'Nl|yn)  ‘  pl(N1lyn,)1* 


We  now  simplify  (6.11a)  by  a  series  of  replacements,  each  one  altering  the 

term  by  o(c).  First  replace  all  the  y£  by  y.  By  Theorem  3  and  |?Se  ,  -UC I  «  0(/e) 

n  n+1  n1 

and  a  differentiability  argument  such  as  used  below  (5.5),  this  only  alters  the 
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2  —  ^  _ 

term  by  o(e).  Since  v^P  (N^lyl-v^P  (N^ly)  =  0,  we  delete  thi8  part  of  the 


result¬ 


ing  summand.  We  now  have 


(6.11b)  T.  +  T,  =  o ( e )  +  /e  lb  (Uf)  f  (UL,n£)]  EG(UEx,-U£)  J  q'  , 

23  N  n  u  n  u  n  n+1  n  .  u ,  ,  -j 

3-n+l  J 

where  for  j  n+1, 

qj  =  (v2p2(^n+l,;i_n"1'N2^y>'Ulpl(^n+l'^"n“1'Nl^)  =^n+l6uj(y) 

Finally,  by  the  differentiability  result  of  Theorem  3,  (6.11b)  equals 

00 

(6.12)  T  +  T  =  o(e)  +  ebM(U£)  [h  ,(U£)f  (UE,ne))  V  E^u  (y)6u.(y). 

2  3  NnNnun  u.L,nn/  i 

3=n+l 


The  difference  between  (6.11b)  and  (6.12)  is  simply  due  to  whether  y£  or  y  is 

.  'Ve 

used  to  get  X  and  U  .  from  X  and  U  . 

n+1  n+1  n  n 


Part  3.  Now,  we  "average  out"  the  sum  in  (6.12).  Define  f^tnE)  by 


f2(nc)  =  ebN(UG)  (bN(Un)fu(Un,ne)  ju  \  l  [E^u  (y)  (y)  -E<5u  (y)  61^  (y)  ] 

j*n  k=j+l  J  J  - 


By  the  (uniform)  geometric  convergence  result  of  Theorem  2,  the  sum  converges 
absolutely  and  |fG(ne) |  ■  0(e) .  By  a  straightforward  calculation  using  the 
stationarity  of  {5u^(y) )  under  P,  we  can  show  that 


&£f£ (ne+t) -f£ (ne)  =  -  (6.12)  +  ote) 
n  2  2 


*  <v86|bN<S‘>f  <S',nCnu  l  EHu0iy)Su  (5>) 

1*1 


Finally,  we  treat  the  term  before  the  o(e)  of  (6.7)  -  in  the  form  in 
which  it  is  written  below  (6.7) .  Define  f£(ne)  by 


By  a  procedure  similar  to  that  used  for  it  can  readily  be  shown  that 

f  (UE ,ne) 

,  ,e ,  .  .  ,  uu  n  .2.?;e.—  /x  ,->>2 

E  f  (ne+e)-f  (ne)  =  o(e)  +  e  - - -  bx,(U  )E(6u  (y) ) 

n  u  u  i.  N  n  u 

f  (U£,ne)  _  _ 

-  e  - bM{Un,^Ia{5,Jln(5,+6(?)J2n<5)J  ' 


Summarizing  the  previous  calculations 


Eefe (ne+e) -fe (ne)  =  o(e)  +  ef  (uE,ne)  +  ef  (UE  ,ne)GUEb.,(UE) 
n  tn  unnNn 


rvf'  Of  r»  —  —  — 

(6.14)  +  efu(Un,ne)bNfU(Un)bN(Un)  lE  6uQ (y) 6u. (y) 

f  (uE,ne)  _  _  „  ®  _ 

+  e  -UU  / -  b^(UE)  [E(6uq  (y) ) 1  +  2  \  e  6uq  (y)  «Uj  (y)  ] 


Part  4 .  Conclusion.  Reintroduce  the  superscript  N.  Fix  N.  All  the 
fE'^  are  bounded  and  of  order  0(>/e)  and  {UE,^J  =  {UE'^(0)}  is  tight.  Also 

l  0 

EE'NfG,N(ne+e)-fE'N(ne)  *  0(e).  Thus,  by  [7],  Theorem  2, 
n 

the  bounded  sequence  {U  '  (•)}  is  tight  in  D[0,®).  Let  e  index  a  weakly  con¬ 
vergent  subsequence  with  limit  U1^ ( - )  .  Since  A  is  defined  to  be  the  infinites¬ 
imal  operator  of  (6.3),  by  (6.14)  and  Theorem  1,  we  see  that  U**(*)  solves  the 

N 

martingale  problem  corresponding  to  an  infinitesimal  operator  A  whose  coeffic 

ients  equal  those  of  A  in  S„.  Thus,  by  Theorem  1,  {u  (•)}  converges  weakly  to 

N 

solution  u(-)  of  (6.3).  The  independence  of  B(.)  and  u(0)is  a  consequence  of 


the  fact  that  &(•)  is  the  unique  solution  to  the  martingale  problem.  The 
stationarity  aasertion  is  not  hard  to  prove,  but  we  omit  the  details.  Q.B.D. 


VII.  ASYMPTOTIC  THEORY  OF  AN  ADAPTIVE  QUANTIZER:  INTRODUCTION 
In  recent  years  there  has  been  a  great  deal  of  effort  concerning  the  effic¬ 
ient  quantization  of  signals  in  telecommunications  systems,  e.g.  of  voice 
signals  in  telephone  transmission  systems.  Let  z ( - )  denote  the  actual  signal 
process  and  A  a  sampling  interval.  In  the  problem  of  interest,  the  signal  is 
sampled  at  moments  {nA,  n*0,l,...},  then  the  samples  {z(nA)}  are  quantized,  and 
it  is  only  the  quantized  samples  which  are  transmitted.  Let  0  *  £  <  <  . . .  < 

5l_1  <  =  «,  0  =  r'1  <  h2  ...  <  Hj  where  ni+1»  i  =  0,  ...,  L-l,  are  real 

numbers.  Let  the  quantization  function  Q(*)  be  defined  as  follows:  there  is 
a  y  >  0  such  that  for  z(nA)  >  0,  Q(z(nA))  =  yr^  if  z(nA)  £  ly£^_^ »  and  set 
g(-z)  =  -Q(z) .  The  parameter  y  is  a  scaling  parameter.  As  the  signal  power 
increases  (decreases) ,  y  should  increase  (decrease)  for  efficient  reconstruction 
of  the  signal  from  the  sequence  of  quantizations. 

The  problem  of  choosing  appropriate  values  of  y  when  the  signal  powers  can 
vary  by  an  order  of  magnitude  or  more  has  led  to  the  study  of  adaptive  quan¬ 
tizers.  We  give  only  a  brief  description  in  order  to  formulate  the  problem. 

For  more  detail  and  discussion  of  the  engineering  considerations,  the  reader 
is  referred  to  the  references  [4],  [5].  Let  e  denote  a  "rate  of  adjustment" 
parameter  for  the  scale  parameter  y  and  let  y^  denote  the  value  of  the  adapted 
scale  parameter  at  the  nth  sampling  instant.  Set  6  £  (0,1]  and  let  0  <  < 

<  1,  >  1.  We  study  an  adaptive  quantizer  which 

is  a  truncated  form  of  the  (typical  in  such  an  application)  adaptive  system 

(7-1)  y‘+1  =  lyCn'\>  "here  B*  «  if  |z(nA)|  £  . 


M2  <  ‘  ‘  *  <  <  °°  w:*-th 


Goodman  and  Gersho  (4)  did  a  thorough  analysis  of  (7.1)  for  the  case  8  • 

1  and  {z(nA)}  independent  and  identically  distributed.  With  8  <  1,  the  system 
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has  some  desirable  robustness  properties  and  this  case,  together  with  simula¬ 
tions,  is  discussed  by  Mitra  [5]  and  others.  The  last  reference  is  concerned 
more  with  reconstruction  of  the  process  z(-)  from  (Q(z(nA))!  and  does  not  give  an 
asymptotic  analysis. 

Generally,  with  non-i.i.d.  {z(nA)},  it  is  hard  to  get  concrete  information 

on  { y e >  for  large  n.  If  the  signal  power  varies  over  time  or  if  (as  is  real- 

istic  for  moderate  values  of  A)  (z(nA) }  is  not  i.i.d.,  then  techniques  such 

as  used  in  [4]  fail,  but  for  small  rates  of  adjustment  (e)  an  asymptotic 

analysis  can  still  shed  light  on  the  process  behavior.  At  the  present  time, 

it  seems  that  little  more  can  be  done  for  the  general  case.  Here,  we  scale  the 

problem  so  that  an  asymptotic  analysis  is  possible.  For  mathematical  as  well 

as  practical  purposes,  it  is  useful  to  confine  y^  to  some  finite  positive 

interval  [y^,y  ]•  Now,  we  define  the  truncated  form  of  (7.1)  which  will  be 

studied.  Let  a  >  0,  0  <  ae  <  1  and  let  {J^}  be  real  numbers  such  that  < 

St  <  ...  <  K  and  l,  <0,  JL  >0.  Then  we  use 
2  L  J. 


(7.2) 


n+1 


'l 


where  I  denotes  truncation  and 


BE  =  (l+ei.^)  if  |  z  (nA)  j  G  ly^i-1  ,yn^’i)  " 


The  asymptotic  results  can  be  used  to  get  information  on  the  effects  of  the 
{ e ^ } ,  A,  structure  of  z(-)  and  a  on  the  performance  for  small  e.  For  notational 
convenience  below,  let  y^  <  1  and  y^  >  1.  Rewrite  (7.2)  in  the  form  (7.3),  where 
y1  •  y[l-ea  log  y]+0(e2)  and  (l+cbE)  =  BE  are  used,  and  F  and  bE  have  the 


obvious  definitions. 


-27- 


(7.3)  y 


n+1 


.  e  , ,  ,  e .  c,  €  _  ,  2 .  . 

[y  (1+eb  )-euy  loq  y  +0(e  )] 
n  n  n  n 


ly  +i  K(y'  ,  z(nA) )  +0(e2)  ] 
n  n 


In  [4],  the  process  (log  y£}  rather  than  {y£}  is  dealt  with. 

n  n 

We  proceed  in  very  much  the  same  way  that  we  did  for  the  automata  problem. 
The  main  difference  arises  from  the  unboundedness  of  Iz(nA)},  under  assumption 
(7.6) .  By  definition, 


L 

=  l 

i=l 


l .  I  {  z  ( nA ) 

l  ' 


£ 

There  are  continuous  functions  f  (•)  such  that  (7.4)  and  the  properties  below 
it  hold. 


(7.4) 


e  e  . ,  „e .  e  . 

y  .  =  y  (1+eB  (y  ) ) 

n+1  n  n  Jn 


e  e  2 

cay  log  y  +  0(£  ) 
n  ■'n 


=  y£  +  ef  (ye,z(nA))  +  0(e2)  , 
n  e  n 


where 

(7.5)  B£(y)  =  l  *£(y)I-,|z(nA>  |  €  ty^.^^y^)  . 

i*l 

£  £ 

Also,  £^(*)  can  be  chosen  such  that  fc^(-)  =  out  of  an  0(e)  neighborhood  of 

y ^  (resp.  yy)  if  t,  <  0  (resp.  JL  >  0),  and  0  £*T(y)  >_  JL  for  <  0  and 

0  <  eE(y)  <  for  l,  >  0. 

~  1  —  1  1 
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Some  assumptions . 
Gaussian  process  with  a 
ically  stable  matrix  M, 


For  specificity,  z(-)  is  assumed  to  be  a  stationary 
rational  spectral  density.  Thus  there  are  an  asymptot- 
a  matrix  C,  a  row  vector  D,  and  a  process  v(>)  such  that 


(7.6)  dv  =  Mvdt  +  Cdw 

z  =  Dv,  w(-)  =  vector-valued  standard  Brownian  motion. 


This  assumption  is  not  essential  -  only  certain  smoothness  properties  of 
the  multivariate  density  are  used,  together  with  the  exponential  rate  of 
decrease  of  the  effects  of  the  initial  conditions. 

Define  My)  =  EF£  (y,z(nA)  )  and  F(y)  =  EF(y ,  z(nA)  )  .  Let  =  var  z(t)  . 
We  have  (the  subscript  y  denotes  the  derivative) 


(7.7) 


i-  (hn, 

dy  y 


l  e*P  -  ~ 


f2  2 


T  ~  ei-i  exP  ' 


/2noQ  i=l  *  2o 

i  (v£i+1Ki exp  -*>Y/2ao  - a/y- 

v2tto„  i=1 


r  2  2 
€i-ly 

20? 


1  -  a/y 


We  can  see  from  the  terms  in  (7.7)  that  F(y)/y  is  the  sum  of  two  strictly  convex 

functions,  the  first  being  bounded  and  having  a  negative  slope,  and  the  second 

going  to  “  as  y  -*•  0  and  to  as  y  -*■«>.  Thus  there  is  a  unique  y  €  (0,»)  such 

that  Fy(y)  =  0.  Also  £(y)  >  0  for  0  <  y  <  y  and  F(y)  <  0  for  y  >  y  and  F^(y) 

#  0.  We  assume  that  y  £  (y^y^)  .  For  small  e,  the  assertions  in  the  last 

sentence  hold  with  F  replacing  F.  Define  UE  -  (y£-y)//e  and  let  E  denote 

r.  n  n  n 

expectation  conditioned  on  {v(jA),j<n(. 
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VIII.  TIGHTNESS  OF  (uS  SMALL  c,  LARGE  n) 

_ n _ 

The  proof  is  similar  to  that  of  Theorem  4  in  Section  V  and  we  oniy  set 

it  up  and  indicate  how  to  deal  with  the  fact  that  Iz(nA) }  is  unbounded. 

Theorem  6 .  Under  the  conditions  in  Section  VI I ,  the  conclusions  of  Theorem  4 
hold . 
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Next,  define  V^(n)  by  V^(n)  =  V]^n,yn)  '  where 

oo  l 

(8.3)  V^(n,y)  =  2e  (y-y)  £  I  £  yiL  [P{y£ . _j<J z( jA) | <y£A | v(nA-A) } 

j=n  i=l 

-  P(y£i_1£| z(nA) | <yCi) ] . 

|v^(n)  |  can  be  estimated  by  use  of  the  following  fact.  There  are  <  *  and 

a  >  0  such  that  |eMt|  <_  K^e  at.  There  is  an  a^  >  0  and  a  <  00  such  that  for 

"aTl/2 

T  >  T  >  0  and  on  the  set  {v(t) :  |v(t)|e  <L } , 

-a  t 

(8.4)  |p{v(t+Ti)  €  B.,  i=l,2|v(t)}  -P(v(t+T.)£B.,  i=l  ,2}  |  <  ^e  . 

for  all  ,  B2. 

In  order  to  use  (8.4)  (in  this  application  we  set  B2  *  range  space  of  v(t)), 
write  the  sum  in  (8.3)  as 

H  00 

(8.5)  l  +  l  , 
j=n  j=H+l 

where  H  =  mintm:  e  n^a/^  |  v(nA-A)  |  <l)  =  0(l+max(0,  log|  v(nA-A)  | ) )  .  Then 
the  first  sum  in  (8.5)  is  0(l+max(0,  log j v (nA-A) | ) ) ,  and  the  second  is  0(1)  by 
(8.4)  and  the  summability  of  £_.>0exp  -a^jA.  Thus  |v^(n)|  = 

0(e)  [l+max(0,  log| v(nA-A)  |)  )  <_  0(e)  (l+|v(nA-A)  |)  .  From  this  point  on,  the 


proof  is  exactly  the  same  as  that  for  Theorem  4. 


Q.E.D. 
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IX.  THE  LIMIT  THEOREM 

%£  've 

We  continue  to  use  the  tilde  'v-  terminology  of  Section  VI,  and  define  Un*  yn» 

EK,  etc.,  as  there.  Also,  set  z(nA)  =  z  (n  i+nl)  and  v(nA)  -  v^A+nA).  The  idea 
n  c 

now  is  still  to  prove  weak  convergence  of  UE(-)-  We  use  E£  for  expectation  condi¬ 


tional  on  (v(jA),  j<n+n£}. 


have  ((9.1b)  defines  y 


^e,N  ^e,N  _  /ve,N 

y_  bV  b 


-y)//T) 


(9.1a) 


'vf  'ur  /—a  , —  'br  'v  a  • 

U  =  U  +  nF  (y  )  +  vc(F  (y  ,z(nA))-F  (y  ))  +  0(e" 
n+1  n  e  n  c  n  e  n 


(9.1b)  U6'"  =  U£,N  +  ft  (F  (yC,N)  +  (F  (y£,N,z(nA))  -  F  (y£'N))  +  0(£3/2).)b  (U£,N> 

n+1  n  e7n  eJn  t  J n  N  n 


Theorem  7 .  Under  the  conditions  of  Section  VII,  the  conclusions  of  Theorem  5 
hold,  but  where  G  =  F^(y)  and  (stationary  process  z(-)  used) 

GO 

a2  =  EF2 (y , z (0) )  +  2  £  EF(y,z(nA) )F(y,z(0) ) . 

n=l 

Remark .  If  M,  c  or  D  were  time-varying,  then  an  extension  of  the  tech¬ 
nique  is  possible,  provided  that  the  tj.me  variation  per  step  is  0(e)  .  The  limit 
diffusion  yields  information  on  the  dependence  of  the  performance  on  the  para¬ 
meters  a,  {£^1,  A,  {£^},  as  well  as  an  estimate  of  the  asymptotic  variance  and 

correlation  function  for  small  t. 

* 

Proof.  Except  for  the  unboundedness  of  the  noise  {z(nA)},  the  proof 
would  be  essentially  the  same  as  that  of  Theorem  5,  and  only  an  outline  will 
be  given. 

Owing  to  the  truncation  |u£,N|  _<  N+1,  the  F^ ,  F£  in  (9.1b)  can  be  replaced  by 

F  and  F,  respectively,  without  changing  the  values,  for  small  e.  Let  us  make 

.2 2  3 

the  replacement.  Fix  f (•,•)€  * Q '  .  Drop  the  superscript  N  on  all  variable* 
for  notatlonal  convenience,  as  done  in  Theorem  5.  Then,  by  a  Taylor  expansion, 
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(9.2)  E£  f  ( tJ£  , ,nt+. )-f (U£,ne>  -  o  (e)  +ef  t  (U£  ,nt)  +ef  (UL,nc)F  (9)UEb,(UC) 
n  n+1  n  tn  un  y  '  n  N  n 


+  /ef  (U£,ne)E£ [F(y+/Tu£,z(nA) )-F(y+/eU£) ]b(U£) 
un  n  n  nNn 


+  f  f  (U£,ne)E£[F(y+/Eu£,z(nA))-F(y+/Eu£)]V(Ue))  . 

2uun  n-'n  'nNn 

Since  the  second  derivative  of  E^F(y,z(nA))  with  respect  to  y  is  bounded  by 

i  ^  i 

constant  [1+ 1  v  (nA-A)  ],  the  next-to-last  term  of  (9.2)  can  be  written  as 


(9.3)  /ef  (U£ , ne) E£ [F (y , z(nA) ) -F (y) ]b  (U&) 
u  n  n  1  N  n 


+  efu(U£,ne)|^E£ [F(y, z(nA) ) -F(y) ] 


U£b ,(U£)+o(e)  [1+ | v (nA-A)  | ] 
-  n  N  n 

y=y 


The  last  term  of  (9.2)  can  be  written  as  (recall  that  F(y)  =  0) 


(9.4)  f  (U£ ,ne) E£ [F(y , z(nA) ) -F(y) ] 2fc>2 (U£)  +  o(e). 

2uun  n  N  n 


Now,  we  use  the  method  of  Theorem  5  in  order  to  average  out  the  terms  of 

(9.2).  We  use  f£(ne)  =  f(UC,ne)+  Y6  ,f£(ne).  Define  f£(ne)  by  (to  average 

n  ‘■1=3  l  3  J 

out  the  second  term  of  (9.3)) 


f£(ne)  =  ef  (U£,ne)b  (U£)U£ 
3  u  n  N  n  n 


Y  — 
.L  3y 
3=n 


E£ [F(y,z( jA) )-F(y) ] 


y=y 


By  an  argument  similar  to  that  used  below  (8,5),  together  with  the  derivative 

bound  stated  above  (9.3),  it  can  be  shown  that  ECf £ (nc+e) -f £ (ne)  =  -  (second 

n  J  3 

term  of  (9.3))  +  o(i)|l  +  |v(nA-A)|2J  and  that  |f£(nc)|  <  0(c) 11+  |v(nA-A)|). 
Next,  introduce  f^(nt)  (to  average  out  (9.4)): 
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f4(n£)  =  f  fuu(“n,ne)bN(*n)  ^  l^F2  (y  ,  z  (  jA)  ) -EF2  (y  ,  z  (  jA)  )  )  . 

j=n 

Then,  as  for  f*,  we  have  |fj(nc)|  1  0(e) [1  +  |v(nA-A)|).  Using  this,  it  is  not 
hard  to  show  via  a  small  amount  of  manipulation  that 


Ef  f\ (ne  +  e) -fl  (nt ) 
n  4  ^ 


L-  f  (IJ1  ,nc)b  ,(U  )  IE  F  (y,z(nA) ) -EF  (y,z(nA))l 
7  uu  n  N  n  n 


+  o(e)  U  +  v(nA-A) | ) . 


Next,  introduce  f^(ne)  in  order  to  average  out  the  first  term  of  (9.3): 


f^(ne)  =  /e  f  (U£,ne)b  <uS  j>  E^F(y,z( jA) ) - 
5  un  Nn.  n 

3=n 


Then,  again,  |f^(ne)|  =  o(/e)(l  +  |v(nA-A)|)  and  we  can  write 


(9.5a)  E^f ^(ne+e) -f ^ (ne) 


-  (first  term  of  (9.3) 


+  ^n[fu^n+l'-)bN<&n+l)-f„^n'->V^]  l  >  • 

j*n+l 

With  a  small  amount  of  manipulation,  we  can  show  that  the  last  term  of  (9.5a)  equals 


(9.5b) 


ebN(^n)  tfu(^n'ne)bN(^n,]u  l  ^F(?,z( jA) )F(y,z(nA) )+o(e) [1+ |?(nA-A) | ] . 

j-n+1 


Finally,  la  Introduced  In  order  to  avorago  out  the  aum  term  in  (9.5b)  in 

the  same  way  that  f'2<nt)  was  used  to  average  out  (6.12)  in  Theorem  5.  Define 
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(9.6)  fUnc)  =  E[f  (U£,nc)b  (UC)  ]  b,(Ue) 
6  un  NnuNn 


•  £  J  lEk  F(y ,  z (kA) ) E(y , z( jA) )  -KP  ( y ,  z  ( kA )  )  F (y ,  z  (  jA)  )  )  . 
j=n  k~j U  n 


By  (8.4) 


f £  ( nt ) 
6 


is  well  defined  and  is  0(e)  11  + 


| v (nA-A)  I2)  , 


as  will  now  be 


proved . 


e 

Define  H  as  below  (8.5)  and  let  denote  the  (j,k)th  summand  in  (9 


and  write  the  sum  in  (9.6)  as 


H 

I 


j=n 


OO 


I 

k=  j+1 


'v e 

E  B 
n 


e 

j* 


oo 

-  I 

j=H+l 


00 


l 

k=  j  +  1 


M  E 

E  B 
n  3k 


I  +  II. 


By  the  argument  connected  with  (8.5),  the  inner  sum  in  I  is  bounded  by 


(9.7)  l 
k= j+1 


1  'v-e  e 

E  B 
n  ]k 


I  l 

k=  j+1 


'ic  1  'le  1 

E  E.B.J 
n  1  ]k 


O (1  +  | v(nA-A) |) . 


Thus,  by  the  bound  on  (H-n),  1  j<  o(l  +  |v(nA-A)  |2)  .  To  treat  II,  we  note  the 

following:  there  is  a  K  <  00  such  that  for  H  <  j  <  k,  |eEBL  |  <_  K  exp  -a  (j- 

^  n  jk  2  1 

Also ,  for  k  >  j, 


&EF(y,z(kA))  ^  K2 [exp  -a1(k-j)A+l{ j (exp  -a(k-j) A) v( jA-A) |  >  1 } 1 . 


With  a  little  more  work,  these  estimates  yield  the  existence  of  a  <  »  such 
that  Ie^B^I  <_  (1+0  (  |  v(nA-A)  | )  exp  -a^A  [  ( j-n)  +  (k-  j)  ]/2 ,  from  which  the  fact 
that  II  =  0(1)  and  the  last  sentence  of  the  previous  paragraph  both  follow. 

It  is  straightforward  to  show  that 


n)  A . 


i 
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lV  f  (  nt  +f  )  -  f'  (nr)  =  -(sum  term  in  (0.5)) 


+  r-bfjdJn)  [fu(Un'ne)bN(Un)  5U  I  EF(y,z(nA)  ) F(y , z(0)  )  +  o(c)  [1+  |  v(nA-A) 

n=l 


Summarizing,  with  f  (nt)  defined  by  f  (ne)  f(U^,ne)  +  £  (nt :) , 


have 


(9  .  S)  Ef  fl  (ne  +  el-f1' (ne)  =  o(e)  [l+|v(nA-A)  |")  +  ef(Ul,nc)  +  ef  (lT,nr)F  (yJlTbflT) 
n  tn  uny  nNn 


o-r. 


'vt.  .'ye. 


+  eb  (U£)  [f  (uc,ne)b  (Ue)  ] 
N  n  u  n  N  n 


O-r 

,  ,je, fuu(Un'ne)  „„2 

+  £bN(V - 2—  EF  (y' 


l  EF ( y , z ( nA )  )  F  ( y  ,  z  ( 0 )  ) 
U  n=l 

z(0) ) +o (e)  [1+ | v(nA-A)  |  2)  . 


Now,  if  the  {uE'N(’)}  (returning  to  the  use  of  superscript  N)  were  tight 
for  each  N,  then  (9.8)  and  Theorem  1  imply  that  any  weakly  convergent  sub¬ 
sequence  of  {U£'V)>  converges  to  a  diffusion  with  operator  AN,  whose  coeffic- 

ients  equal  those  of  A  in  S.,  and,  hence,  that  the  original  lu  (•)}  converge 

N 

weakly  to  the  solution  of  (6.3)  with  the  G  and  o  defined  in  Theorem  7. 

But  (dropping  the  superscript  N  again)  |£b_3f^(ne)|  =  0  ( /c)  H+|v(nA-A)  |  1 

and  | EG f G (nc+E) -fE (ne)  1  =  0(e)  +  o ( e)  (1+ | v(nA-A)  | 2]  and  for  any  T  <  »  , 
n 

K  >  0,  the  Gaussian  property  implies  that 

lim  P{sup  elv(ne)  |2  £  K}  =0. 
r-K)  n<T/r 

Thus,  tightness  follows  by  Theorem  2  of  (1]  or  [71,  as  it  did  for  the  case  of 


Theorem  1 . 


Q.E.D. 
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